Enhancing Real-time Embedded Image Processing Robustness on Reconfigurable Devices for Critical Applications

Nowadays, image processing is increasingly used in several application fields, such as biomedical, aerospace, or automotive. Within these fields, image processing is used to serve both non-critical and critical tasks. As example, in automotive, cameras are becoming key sensors in increasing car safety, driving assistance and driving comfort. They have been employed for infotainment (non-critical), as well as for some driver assistance tasks (critical), such as Forward Collision Avoidance, Intelligent Speed Control, or Pedestrian Detection. The complexity of these algorithms brings a challenge in real-time image processing systems, requiring high computing capacity, usually not available in processors for embedded systems. Hardware acceleration is therefore crucial, and devices such as Field Programmable Gate Arrays (FPGAs) best fit the growing demand of computational capabilities. These devices can assist embedded processors by significantly speeding-up computationally intensive software algorithms. Moreover, critical applications introduce strict requirements not only from the real-time constraints, but also from the device reliability and algorithm robustness points of view. Technology scaling is highlighting reliability problems related to aging phenomena, and to the increasing sensitivity of digital devices to external radiation events that can cause transient or even permanent faults. These faults can lead to wrong information processed or, in the worst case, to a dangerous system failure. In this context, the reconfigurable nature of FPGA devices can be exploited to increase the system reliability and robustness by leveraging Dynamic Partial Reconfiguration features. The research work presented in this thesis focuses on the development of techniques for implementing efficient and robust real-time embedded image processing hardware accelerators and systems for mission-critical applications. Three main challenges have been faced and will be discussed, along with proposed solutions, throughout the thesis: (i) achieving real-time performances, (ii) enhancing algorithm robustness, and (iii) increasing overall system's dependability. In order to ensure real-time performances, efficient FPGA-based hardware accelerators implementing selected image processing algorithms have been developed. Functionalities offered by the target technology, and algorithm's characteristics have been constantly taken into account while designing such accelerators, in order to efficiently tailor algorithm's operations to available hardware resources. On the other hand, the key idea for increasing image processing algorithms' robustness is to introduce self-adaptivity features at algorithm level, in order to maintain constant, or improve, the quality of results for a wide range of input conditions, that are not always fully predictable at design-time (e.g., noise level variations). This has been accomplished by measuring at run-time some characteristics of the input images, and then tuning the algorithm parameters based on such estimations. Dynamic reconfiguration features of modern reconfigurable FPGA have been extensively exploited in order to integrate run-time adaptivity into the designed hardware accelerators. Tools and methodologies have been also developed in order to increase the overall system dependability during reconfiguration processes, thus providing safe run-time adaptation mechanisms. In addition, taking into account the target technology and the environments in which the developed hardware accelerators and systems may be employed, dependability issues have been analyzed, leading to the development of a platform for quickly assessing the reliability and characterizing the behavior of hardware accelerators implemented on reconfigurable FPGAs when they are affected by such faults

The Virtues of Frugality - Why cosmological observers should release their data slowly

Cosmologists will soon be in a unique position. Observational noise will gradually be replaced by cosmic variance as the dominant source of uncertainty in an increasing number of observations. We reflect on the ramifications for the discovery and verification of new models. If there are features in the full data set that call for a new model, there will be no subsequent observations to test that model's predictions. We give specific examples of the problem by discussing the pitfalls of model discovery by prior adjustment in the context of dark energy models and inflationary theories. We show how the gradual release of data can mitigate this difficulty, allowing anomalies to be identified, and new models to be proposed and tested. We advocate that observers plan for the frugal release of data from future cosmic variance limited observations.Comment: 5 pages, expanded discussion of Lambda and of blind anlysis, added refs. Matches version to appear in MNRAS Letter

AIDI: An adaptive image denoising FPGA-based IP-core for real-time applications

The presence of noise in images can significantly impact the performances of digital image processing and computer vision algorithms. Thus, it should be removed to improve the robustness of the entire processing flow. The noise estimation in an image is also a key factor, since, to be more effective, algorithms and denoising filters should be tuned to the actual level of noise. Moreover, the complexity of these algorithms brings a new challenge in real-time image processing applications, requiring high computing capacity. In this context, hardware acceleration is crucial, and Field Programmable Gate Arrays (FPGAs) best fit the growing demand of computational capabilities. This paper presents an Adaptive Image Denoising IP-core (AIDI) for real-time applications. The core first estimates the level of noise in the input image, then applies an adaptive Gaussian smoothing filter to remove the estimated noise. The filtering parameters are computed on-the-fly, adapting them to the level of noise in the image, and pixel by pixel, to preserve image information (e.g., edges or corners). The FPGA-based architecture is presented, highlighting its improvements w.r.t. a standard static filtering approac

SA-FEMIP: A Self-Adaptive Features Extractor and Matcher IP-Core Based on Partially Reconfigurable FPGAs for Space Applications

Video-based navigation (VBN) is increasingly used in space applications to enable autonomous entry, descent, and landing of aircrafts. VBN algorithms require real-time performances and high computational capabilities, especially to perform features extraction and matching (FEM). In this context, field-programmable gate arrays (FPGAs) can be employed as efficient hardware accelerators. This paper proposes an improved FPGA-based FEM module. Online self-adaptation of the parameters of both the image noise filter and the features extraction algorithm is adopted to improve the algorithm robustness. Experimental results demonstrate the effectiveness of the proposed self-adaptive module. It introduces a marginal resource overhead and no timing performance degradation when compared with the reference state-of-the-art architecture

SATTA: a Self-Adaptive Temperature-based TDF awareness methodology for dynamically reconfigurable FPGAs

Dependability issues due to non functional properties are emerging as major cause of faults in modern digital systems. Effective countermeasures have to be presented to properly manage their critical timing effects. This paper presents a methodology to avoid transition delay faults in FPGA-based systems, with low area overhead. The approach is able to exploit temperature information and aging characteristics to minimize the cost in terms of performances degradation and power consumption. The architecture of a hardware manager able to avoid delay faults is presented and deeply analyzed, as well as its integration in the standard implementation design flow

Dependable Dynamic Partial Reconfiguration with minimal area & time overheads on Xilinx FPGAS

Thanks to their flexibility, FPGAs are nowadays widely used to implement digital systems' prototypes and, more frequently, their final releases. Reconfiguration traditionally required an external controller to upload contents in the FPGA. Dynamic Partial Reconfiguration (DPR) opens new horizons in FPGAs' applications, providing many new utilization paradigms, as it enables an FPGA to reconfigure itself: no external controller is required since it can be included in the FPGA. However, DPR also introduces reliability issues related to errors in the partial reconfiguration bitstreams. FPGA manufacturers currently provide solutions that are not efficient. In this paper new DfD (Design for Dependability) techniques are proposed. Exploiting information density of configuration data, they improve the performance while providing the same reliability characteristics as the previous one

FEMIP: A high performance FPGA-based features extractor & matcher for space applications

Nowadays, Video-Based Navigation (VBN) is increasingly used in space-applications. The future space-missions will include this approach during the Entry, Descent and Landing (EDL) phase, in order to increase the landing point precision. This paper presents FEMIP: a high performance FPGA-based features extractor and matcher tuned for space applications. It outperforms the current state-of-the-art, ensuring a higher throughput and a lower hardware resources usage

SAFE: a Self Adaptive Frame Enhancer FPGA-based IP-core for real-time space applications

Video-based navigation is an increasingly used procedure with hard real-time requirements and high computational effort. In this field, FPGA hardware accelleration supplyes low-cost and considerable performances enhancement. The video-based navigation algorithms extrapolate and correlates features from images, relying on their accuracy. Image enhancement provides more defined and contrasted frames, assuring high precision feature extraction. This work introduces an FPGA-based self-adaptive image enhancer. The IP-core is suitable for hard-real time applications, such as space applications, thanks to the guaranteed high-throughput

SAFE: a Self Adaptive Frame Enhancer FPGA-based IP-core for real-time space applications

Video-based navigation is an increasingly used procedure with hard real-time requirements and high computational effort. In this field, FPGA hardware accelleration supplyes low-cost and considerable performances enhancement. The video-based navigation algorithms extrapolate and correlates features from images, relying on their accuracy. Image enhancement provides more defined and contrasted frames, assuring high precision feature extraction. This work introduces an FPGA-based self-adaptive image enhancer. The IP-core is suitable for hard-real time applications, such as space applications, thanks to the guaranteed high-throughpu

Double threading through DNA: NMR structural study of a bis-naphthalene macrocycle bound to a thymine–thymine mismatch

The macrocyclic bis-naphthalene macrocycle (2,7-BisNP), belonging to the cyclobisintercalator family of DNA ligands, recognizes T–T mismatch sites in duplex DNA with high affinity and selectivity, as evidenced by thermal denaturation experiments and NMR titrations. The binding of this macrocycle to an 11-mer DNA oligonucleotide containing a T–T mismatch was studied using NMR spectroscopy and NMR-restrained molecular modeling. The ligand forms a single type of complex with the DNA, in which one of the naphthalene rings of the ligand occupies the place of one of the mismatched thymines, which is flipped out of the duplex. The second naphthalene unit of the ligand intercalates at the A-T base pair flanking the mismatch site, leading to encapsulation of its thymine residue via double stacking. The polyammonium linking chains of the macrocycle are located in the minor and the major grooves of the oligonucleotide and participate in the stabilization of the complex by formation of hydrogen bonds with the encapsulated thymine base and the mismatched thymine remaining inside the helix. The study highlights the uniqueness of this cyclobisintercalation binding mode and its importance for recognition of DNA lesion sites by small molecules